Language-Based Image Editing with Recurrent Attentive Models

We investigate the problem of Language-Based Image Editing (LBIE). Given a source image and a natural language description, we want to generate a target image by editing the source image based on the description. We propose a generic modeling framework for two sub-tasks of LBIE: language-based image segmentation and image colorization. The framework uses recurrent attentive models to fuse image and language features. Instead of using a fixed step size, we introduce for each region of the image a termination gate to dynamically determine after each inference step whether to continue extrapolating additional information from the textual description. The effectiveness of the framework is validated on three datasets. First, we introduce a synthetic dataset, called CoSaL, to evaluate the end-to-end performance of our LBIE system. Second, we show that the framework leads to state-of-the-art performance on image segmentation on the ReferIt dataset. Third, we present the first language-based colorization result on the Oxford-102 Flowers dataset.Comment: Accepted to CVPR 2018 as a Spotligh

Spatial and temporal properties of precipitation uncertainty structures over tropical oceans, The

2015 Spring.Includes bibliographical references.The global distribution of precipitation has been measured from space using a series of passive microwave radiometers for over 40 years. However, our knowledge of precipitation uncertainty is still limited. While previous studies have shown that the uncertainty associated with the surface rain rate tends to vary with geographic location and season, most likely as a consequence of inappropriate and inaccurate microphysical assumptions in the forward model, the internal uncertainty structure remains largely unknown. Hence, a classification scheme is introduced, in which the overall precipitation uncertainty consists of random noise, constant biases, and region-dependent cyclic patterns. It is hypothesized that those cyclic patterns are the result of an imperfect forward model simulation of precipitation variation associated with regional atmospheric cycles. To investigate the hypothesis, differences from ten years of collocated surface rain rate measurements from TRMM Microwave Imager and Precipitation Radar are used as a proxy to characterize the precipitation uncertainty structure. The results show that the recurring uncertainty patterns over tropical ocean basins are clearly impacted by a hierarchy of regionally prominent atmospheric cycles with multiple time scales, from the diurnal cycle to multi-annual oscillation. Spectral analyses of the uncertainty time series have also confirmed the same argument. Moreover, the relative importance of major uncertainty sources varies drastically not only from one basin to another, but also with different choices of sampling resolutions. Following the classification scheme and hypothesis proposed in this study, the magnitudes of un-explained precipitation uncertainty can be reduced up to 68% and 63% over the equatorial central Pacific and eastern Atlantic, respectively

A quantitative structure-activity relationship (QSAR) study of chlorinated cyclodiene insecticide analogs

Quantitative structure-activity relationships (QSAR) between the inhibitory effect of specific t-butylbicyclophosphorothionate (TBPS) binding to rat brain P2 membrane, a lipophilic parameter, and topological indices, were studied for 33 chlorinated alicyclic insecticides such as heptachlor, aldrin and their structural analogs. This study shows that lipophilicity plays an important role in the action of cyclodiene compounds. The epoxide or ketone structural congeners, and the non-epoxide, non-ketone cyclodiene analogs exhibit two different QSARs and may bind to different regions, respectively, at the common GABA receptor. The epoxide or ketone congeners may bind at a slightly more hydrophilic region, and a negatively correlated linear relationship exists between the inhibition of TBPS-binding and lipophilicity. However, the non-epoxide, non-ketone analogs may bind at a very lipophilic region, and there is a positively correlated linear relationship between their binding and their lipophilicity. The epoxide feature of the cyclodienes seems to be an essential structural requirement for eliciting high inhibitory activity at the GABA receptor. Further the dependence of biological activity on structure can be described by a multiple-variate model with a combination of three explanatory variables among the first-, second-, third- and fourth-valence molecular connectivity indices, i.e., [superscript]1[chi][superscript] v, [superscript]2[chi][superscript] v, [superscript]3[chi][superscript] v, and [superscript]4[chi][superscript] v. High correlation coefficients (r = 0.934 to 0.941) between the biological response variable and the explanatory molecular connectivity indices demonstrated that the topological and steric attributes of the cyclodienes are structural characteristics important to for their biological activity. Electronic effects probably also contribute to the toxicity of the cyclodienes, but the parameter selected in the study, i.e., the bridge-carbon protons\u27 chemical shift in NMR spectra, does not reveal any relationship to the TBPS binding;The information drawn from such studies will benefit our understanding of the structural determinants for the biological action of the classic cyclodiene insecticides, and future approaches could be directed at the synthesis of modified cyclodiene-type insecticides, perhaps bearing fewer chlorines in the molecular framework. A better understanding of cyclodiene QSARs will also contribute to an improved capability to assess the toxicological significance of the ubiquitous environmental residues of the cyclodienes and their degradation products

Self-supervised Spatio-temporal Representation Learning for Videos by Predicting Motion and Appearance Statistics

We address the problem of video representation learning without human-annotated labels. While previous efforts address the problem by designing novel self-supervised tasks using video data, the learned features are merely on a frame-by-frame basis, which are not applicable to many video analytic tasks where spatio-temporal features are prevailing. In this paper we propose a novel self-supervised approach to learn spatio-temporal features for video representation. Inspired by the success of two-stream approaches in video classification, we propose to learn visual features by regressing both motion and appearance statistics along spatial and temporal dimensions, given only the input video data. Specifically, we extract statistical concepts (fast-motion region and the corresponding dominant direction, spatio-temporal color diversity, dominant color, etc.) from simple patterns in both spatial and temporal domains. Unlike prior puzzles that are even hard for humans to solve, the proposed approach is consistent with human inherent visual habits and therefore easy to answer. We conduct extensive experiments with C3D to validate the effectiveness of our proposed approach. The experiments show that our approach can significantly improve the performance of C3D when applied to video classification tasks. Code is available at https://github.com/laura-wang/video_repres_mas.Comment: CVPR 201